This is a preliminary evaluation of forecasts made for the German Forecast Hub. These evaluations are preliminary - this means I cannot currently rule out any mistakes and the plots and analyses are subject to change. The evaluations are not authorised by the German Forecast Hub team.

Feel free to reproduce this analysis. To that end, you can clone this repository and the Forecast Hub repository, specify the directory of the submissions files in the R-Markdown script and run the script.

If you have questions or want to give feedback, please create an issue in this repository.

Forecast Visualisation

Germany

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

Poland

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

Score Ranking

Overall

Overall separate

Germany

Poland

Score Overview

Overview of all the models and metrics

Overall

Overall separate

Germany

Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

Poland

Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

Overiew of Scores by Horizon

Overview of all models and metrics stratified by forecast horizons

horizon = 1

Overall

Overall separate

Germany

Poland

horizon = 2

Overall

Overall separate

Germany

Poland

horizon = 3

Overall

Overall separate

Germany

Poland

horizon = 4

Overall

Overall separate

Germany

Poland

Interval Score Components

Weighted interval score broken down into its components “overprediction penalty”, “underprediction penalty” and “sharpness”

Overall

Overall separate

Germany

### Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

Poland

### Overall

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23

Calibration

Interval Coverage

Overall

Overall separate

Plot of the percentage of true values captured by each prediction interval

Quantile Coverage

Overall

Overall separate

Plot of the percentage of true values below each predictive quantile level

Bias

Overall

Overall separate

Separate by date

2020-10-12

2020-10-19

2020-10-26

2020-11-02

2020-11-09

2020-11-16

2020-11-23